Task demand mediates the interaction of spatial and temporal attention.

Psychophysical studies typically test attentional mechanisms in isolation, but in everyday life they interact to optimize human behavior. We investigated whether spatial and temporal attention interact in two orientation discrimination experiments that vary in task demand. We manipulated temporal and spatial attention separately and conjointly with well-established methods for testing each spatial or temporal attention. We assessed sensitivity (d') and reaction time for every combination of spatial and timing cues, each of which was valid, neutral, or invalid. Spatial attention modulated sensitivity (d') and speed (reaction time) across temporal attention conditions. Temporal attention modulated sensitivity and speed under high- but not low- task demands. Furthermore, spatial and temporal attention interacted for the high-demand task. This study reveals that task demand matters; in a simple task spatial attention suffices to improve performance, whereas in a more demanding task both spatial and temporal attention interact to boost performance, albeit in a subadditive fashion.

Temporal attention recruits fronto-cingulate cortex to amplify stimulus representations.

The human brain receives a continuous stream of input, but it faces significant constraints in its ability to process every item in a sequence of stimuli. Voluntary temporal attention can alleviate these constraints by using information about upcoming stimulus timing to selectively prioritize a task-relevant item over others in a sequence. But the neural mechanisms underlying this ability remain unclear. Here, we manipulated temporal attention to successive stimuli in a two-target temporal cueing task, while controlling for temporal expectation by using fully predictable stimulus timing. We recorded magnetoencephalography (MEG) in human observers and measured the effects of temporal attention on orientation representations of each stimulus using time-resolved multivariate decoding in both sensor and source space. Voluntary temporal attention enhanced the orientation representation of the first target 235-300 milliseconds after target onset. Unlike previous studies that did not isolate temporal attention from temporal expectation, we found no evidence that temporal attention enhanced early visual evoked responses. Instead, and unexpectedly, the primary source of enhanced decoding for attended stimuli in the critical time window was a contiguous region spanning left frontal cortex and cingulate cortex. The results suggest that voluntary temporal attention recruits cortical regions beyond the ventral stream at an intermediate processing stage to amplify the representation of a target stimulus, which may serve to protect it from subsequent interference by a temporal competitor.

Spatial attention alters visual cortical representation during target anticipation.

Attention enables us to efficiently and flexibly interact with the environment by prioritizing some image features in preparation for responding to a stimulus. Using a concurrent psychophysics- fMRI experiment, we investigated how covert spatial attention affects responses in human visual cortex prior to target onset, and how it affects subsequent behavioral performance. Performance improved at cued locations and worsened at uncued locations, relative to distributed attention, demonstrating a selective tradeoff in processing. Pre-target BOLD responses in cortical visual field maps changed in two ways: First, there was a stimulus-independent baseline shift, positive in map locations near the cued location and negative elsewhere, paralleling the behavioral results. Second, population receptive field centers shifted toward the attended location. Both effects increased in higher visual areas. Together, the results show that spatial attention has large effects on visual cortex prior to target appearance, altering neural response properties throughout and across multiple visual field maps.

Presaccadic Attention Depends on Eye Movement Direction and Is Related to V1 Cortical Magnification.

With every saccadic eye movement, humans bring new information into their fovea to be processed with high visual acuity. Notably, perception is enhanced already before a relevant item is foveated: During saccade preparation, presaccadic attention shifts to the upcoming fixation location, which can be measured via behavioral correlates such as enhanced visual performance or modulations of sensory feature tuning. The coupling between saccadic eye movements and attention is assumed to be robust and mandatory and considered a mechanism facilitating the integration of pre- and postsaccadic information. However, until recently it had not been investigated as a function of saccade direction. Here, we measured contrast response functions during fixation and saccade preparation in male and female observers and found that the pronounced response gain benefit typically elicited by presaccadic attention is selectively lacking before upward saccades at the group level-some observers even showed a cost. Individual observer's sensitivity before upward saccades was negatively related to their amount of surface area in primary visual cortex representing the saccade target, suggesting a potential compensatory mechanism that optimizes the use of the limited neural resources processing the upper vertical meridian. Our results raise the question of how perceptual continuity is achieved and how upward saccades can be accurately targeted despite the lack of-theoretically required-presaccadic attention.

When temporal attention interacts with expectation.

Temporal attention is voluntarily deployed at specific moments, whereas temporal expectation is deployed according to timing probabilities. When the target appears at an expected moment in a sequence, temporal attention improves performance at the attended moments, but the timing and the precision of the attentional window remain unknown. Here we independently and concurrently manipulated temporal attention-via behavioral relevance-and temporal expectation-via session-wise precision and trial-wise hazard rate-to investigate whether and how these mechanisms interact to improve perception. Our results reveal that temporal attention interacts with temporal expectation-the higher the precision, the stronger the attention benefit, but surprisingly this benefit decreased with delayed onset despite the increasing probability of stimulus appearance. When attention was suboptimally deployed to earlier than expected moments, it could not be reoriented to a later time point. These findings provide evidence that temporal attention and temporal expectation are different mechanisms, and highlight their interplay in optimizing visual performance.

Limited restoration of contrast sensitivity with training after V1 damage in humans.

Stroke damage to the primary visual cortex (V1) causes severe visual deficits, which benefit from perceptual retraining. However, whereas training with high-contrast stimuli can locally restore orientation and motion direction discrimination abilities at trained locations, it only partially restores luminance contrast sensitivity (CS). Recent work revealed that high-contrast discrimination abilities may be preserved in the blind field of some patients early after stroke. Here, we asked if CS for orientation and direction discrimination is similarly preserved inside the blind field, to what extent, and whether it could benefit from a visual training intervention. Thirteen subacute patients (<3 months post-V1-stroke) and 12 chronic patients (>6 months post-V1-stroke) were pre-tested, then trained to discriminate either orientation or motion direction of Gabor patches of progressively lower contrasts as their performance improved. At baseline, more subacute than chronic participants could correctly discriminate the orientation of high-contrast Gabors in their blind field, but all failed to perform this task at lower contrasts, even when 10Hz flicker or motion direction were added. Training improved CS in a greater portion of subacute than chronic participants, but no-one attained normal CS, even when stimuli contained flicker or motion. We conclude that, unlike the near-complete training-induced restoration of high-contrast orientation and motion direction discrimination abilities, V1 damage in adulthood may severely limit the residual visual system's ability to regain normal CS. Our results support the notion that CS involves different neural substrates and computations than those required for orientation and direction discrimination in V1-damaged visual systems.Significance statement Stroke-induced V1 damage in adult humans induces a rapid and severe impairment of contrast sensitivity for orientation and motion direction discrimination in the affected hemifield, although discrimination of high-contrast stimuli can persist for several months. Adaptive training with Gabor patches of progressively lower contrasts improves contrast sensitivity for both orientation and motion discriminations in the blind-field of subacute (<3 months post-stroke) and chronic (>6 months post-stroke) participants; however, it fails to restore normal contrast sensitivity. Nonetheless, more subacute than chronic stroke participants benefit from such training, particularly when discriminating the orientation of static, non-flickering targets. Thus, contrast sensitivity appears critically dependent on processing within V1, with perceptual training displaying limited potential to fully restore it after V1 damage.

Audiovisual integration in the McGurk effect is impervious to music training.

The McGurk effect refers to an audiovisual speech illusion where the discrepant auditory and visual syllables produce a fused percept between the visual and auditory component. However, little is known about how individual differences contribute to the McGurk effect. Here, we examined whether music training experience-which involves audiovisual integration-can modulate the McGurk effect. Seventy-three participants completed the Goldsmiths Musical Sophistication Index (Gold-MSI) questionnaire to evaluate their music expertise on a continuous scale. Gold-MSI considers participants' daily-life exposure to music learning experiences (formal and informal), instead of merely classifying people into different groups according to how many years they have been trained in music. Participants were instructed to report, via a 3-alternative forced choice task, "what a person said": /Ba/, /Ga/ or /Da/. The experiment consisted of 96 audiovisual congruent trials and 96 audiovisual incongruent (McGurk) trials. We observed no significant correlations between the susceptibility of the McGurk effect and the different subscales of the Gold-MSI (active engagement, perceptual abilities, music training, singing abilities, emotion) or the general musical sophistication composite score. Together, these findings suggest that music training experience does not modulate audiovisual integration in speech as reflected by the McGurk effect.

Do microsaccades vary with discriminability around the visual field?

Microsaccades-tiny fixational eye movements- improve discriminability in high acuity tasks in the foveola. To investigate whether they help compensate for low discriminability at perifovea, we examined MS characteristics relative to the adult visual performance field, which is characterized by two perceptual asymmetries: Horizontal-Vertical Anisotropy (better discrimination along the horizontal than vertical meridian), and Vertical Meridian Asymmetry (better discrimination along the lower- than upper-vertical meridian). We investigated whether and to what extent microsaccade directionality varies when stimuli are at isoeccentric locations along the cardinals under conditions of heterogeneous discriminability (Experiment 1) and homogeneous discriminability, equated by adjusting stimulus contrast (Experiment 2). Participants performed a two-alternative forced-choice orientation discrimination task. In both experiments, performance was better on trials without microsaccades between ready signal onset and stimulus offset than on trials with microsaccades. Across the trial sequence the microsaccade rate and directional pattern were similar across locations. Our results indicate that microsaccades were similar regardless of stimulus discriminability and target location, except during the response period-once the stimuli were no longer present and target location no longer uncertain-when microsaccades were biased toward the target location. Thus, this study reveals that microsaccades do not flexibly adapt as a function of varying discriminability in a basic visual task around the visual field.

Featural Representation and Internal Noise Underlie the Eccentricity Effect in Contrast Sensitivity.

Human visual performance for basic visual dimensions (e.g., contrast sensitivity and acuity) peaks at the fovea and decreases with eccentricity. The eccentricity effect is related to the larger visual cortical surface area corresponding to the fovea, but it is unknown if differential feature tuning contributes to this eccentricity effect. Here, we investigated two system-level computations underlying the eccentricity effect: featural representation (tuning) and internal noise. Observers (both sexes) detected a Gabor embedded in filtered white noise which appeared at the fovea or one of four perifoveal locations. We used psychophysical reverse correlation to estimate the weights assigned by the visual system to a range of orientations and spatial frequencies (SFs) in noisy stimuli, which are conventionally interpreted as perceptual sensitivity to the corresponding features. We found higher sensitivity to task-relevant orientations and SFs at the fovea than that at the perifovea, and no difference in selectivity for either orientation or SF. Concurrently, we measured response consistency using a double-pass method, which allowed us to infer the level of internal noise by implementing a noisy observer model. We found lower internal noise at the fovea than that at the perifovea. Finally, individual variability in contrast sensitivity correlated with sensitivity to and selectivity for task-relevant features as well as with internal noise. Moreover, the behavioral eccentricity effect mainly reflects the foveal advantage in orientation sensitivity compared with other computations. These findings suggest that the eccentricity effect stems from a better representation of task-relevant features and lower internal noise at the fovea than that at the perifovea.

Adaptation and exogenous attention interact in the early visual cortex: A TMS study.

Our capacity to process information is constrained by the limited energy available to the brain and the high energy cost of cortical computation [1]. To help manage limited resources and optimize our sensitivity to visual information, adaptation diminishes sensitivity for repetitive stimuli, whereas attention enhances the representation of relevant information [2]. Transcranial magnetic stimulation (TMS) to early visual cortex (V1/V2) eliminates the response-gain effect of exogenous (involuntary) attention on contrast sensitivity: the benefit at the attended location and the cost at the unattended location [3]. Here we investigate whether adaptation modulates the exogenous attentional effect on perception under TMS to V1/V2. Observers performed an orientation discrimination task while attending to one of two stimuli, with or without adaptation. Following a valid, neutral or invalid attentional cue, two cortically-magnified Gabor patches were presented in the stimulated region (matching each observer's phosphene location) and its contralateral symmetric region. A response cue indicated the patch whose orientation observers had to discriminate. The response cue either matched-target stimulated-or did not match- distractor stimulated-the stimulated side. Without adaptation, the exogenous attention response-gain effect emerged in the distractor-stimulated condition-increased contrast sensitivity at the attended location and decreased at the unattended location-but these effects were eliminated in the target-stimulated condition, consistent with our previous findings. Critically, after adaptation, response gain of exogenous attention was observed in both distractor-stimulated and target-stimulated conditions. These results reveal that (1) adaptation and attention interact in the early visual cortex, (2) adaptation shields exogenous attention from TMS effects.

Visual perceptual learning modulates microsaccade rate and directionality.

Microsaccades, incessant "fixational eye movements" (< 1°), are an important window into cognitive functions. Yet, its role in visual perceptual learning (VPL)-improvements in visual discrimination due to practice-remains practically unexplored. Here we investigated whether and how microsaccades change in VPL. Human observers performed a Landolt acuity task for 5 consecutive days and were assigned to the Neutral or Attention group. On each trial, two peripheral Landolt squares were presented briefly along a diagonal. Observers reported the gap side of the target stimulus. Training improved acuity and modified the microsaccade rate; with training, the rate decreased during the fixation period but increased during the response cue. Furthermore, microsaccade direction during the response cue was biased toward the target location, and training enhanced and sped up this bias. Finally, the microsaccade rate during a task-free fixation period correlated with observers' initial acuity threshold, indicating that the fewer the microsaccades during fixation the better the individual visual acuity. All these results, which were similar for both the Neutral and Attention groups and at both trained and untrained locations, suggest that microsaccades could serve as a physiological marker reflecting functional dynamics in human perceptual learning.

Contrast sensitivity: a fundamental limit to vision restoration after V1 damage.

Stroke damage to the primary visual cortex (V1) causes severe visual deficits, which benefit from perceptual retraining. However, whereas training with high-contrast stimuli can locally restore orientation and direction discrimination abilities at trained locations, it only partially restores luminance contrast sensitivity (CS). Recent work revealed that high-contrast discrimination abilities may be preserved in the blind field of some patients early after stroke. Here, we asked if CS for orientation and direction discrimination is similarly preserved inside the blind field, to what extent, and whether it could benefit from a visual training intervention. Thirteen subacute (<3 months post-V1-stroke) and 12 chronic (>6 months post-V1-stroke) participants were pre-tested, then trained to discriminate either orientation or motion direction of Gabor patches of progressively lower contrasts. At baseline, more subacute than chronic participants could correctly discriminate the orientation of high-contrast Gabors in their blind field, but all failed to perform this task at lower contrasts, even when 10Hz flicker or motion direction were added. Training improved CS in a greater portion of subacute than chronic participants, but no-one attained normal CS, even when stimuli contained flicker or motion. We conclude that, unlike the near-complete training-induced restoration of high-contrast orientation and direction discrimination, there is limited capacity for restoring CS after V1 damage in adulthood. Our results suggest that CS involves different neural substrates and computations than those required for orientation and direction discrimination in V1-damaged visual systems.

Poster Session: Evidence for preserved conscious orientation discrimination in perimetrically-blind fields early after V1-damage.

Cortically-blind (CB) patients with stroke damage to the primary visual cortex (V1) lose conscious vision but many exhibit blindsight - the ability to unconsciously detect or discriminate moving or flickering targets inside their blind-fields. However, the prevalence of conscious visual abilities in CB is less clear. Having developed a new method to assess vision inside perimetrically-defined blind fields, we found that >50% of subacute CB patients (<6 months post-stroke) can consciously discriminate global motion inside their blind field. Here, we asked if they can also discriminate orientation of static targets, which do not typically elicit blindsight. In 10 subacute patients, we mapped their intact and blind hemifields using static, non-flickering, 1cpd Gabors across a wide range of luminance contrasts. Blind-field locations were labeled "preserved" if performance was >72.5% correct. Considering overall performance, only 1 participant had preserved static orientation perception in the blind-field. However, this increased to 4 participants when only considering performance at high contrasts (>50%), all of whom reported awareness of stimuli. Thus, early after V1 damage, conscious percepts for oriented, high-contrast, static targets can remain inside CB fields, similar in incidence to global motion discriminations. We are now testing additional patients to assess if these abilities persist into the chronic period and to detail their underlying neural substrates.

Dissociable roles of human frontal eye fields and early visual cortex in presaccadic attention.

Shortly before saccadic eye movements, visual sensitivity at the saccade target is enhanced, at the expense of sensitivity elsewhere. Some behavioral and neural correlates of this presaccadic shift of attention resemble those of covert attention, deployed during fixation. Microstimulation in non-human primates has shown that presaccadic attention modulates perception via feedback from oculomotor to visual areas. This mechanism also seems plausible in humans, as both oculomotor and visual areas are active during saccade planning. We investigated this hypothesis by applying TMS to frontal or visual areas during saccade preparation. By simultaneously measuring perceptual performance, we show their causal and differential roles in contralateral presaccadic attention effects: Whereas rFEF+ stimulation enhanced sensitivity opposite the saccade target throughout saccade preparation, V1/V2 stimulation reduced sensitivity at the saccade target only shortly before saccade onset. These findings are consistent with presaccadic attention modulating perception through cortico-cortical feedback and further dissociate presaccadic and covert attention.

The Extrafoveal Preview Effect is More Pronounced at Polar Angle Locations Where Perception is Poor.

The pre-saccadic preview of a peripheral target enhances the efficiency of its post-saccadic processing, termed the extrafoveal preview effect. Peripheral visual performance -and thus the quality of the preview- varies around the visual field, even at iso-eccentric locations. To investigate whether these polar angle asymmetries influence the preview effect, we asked human participants (N=14) to preview four tilted Gabors at the cardinals, until a central cue indicated to which one to saccade. During the saccade, the target orientation either remained or was flipped (valid/invalid preview). After saccade landing, participants discriminated the orientation of the (briefly presented) second Gabor. Gabor contrast was titrated with adaptive staircases. Valid previews increased participants' post-saccadic contrast sensitivity. This preview effect was inversely related to polar angle perceptual asymmetries; largest at the upper, and smallest at the horizontal meridian. Our finding reveals that the visual system compensates for peripheral asymmetries when integrating information across saccades.

Featural representation and internal noise underlie the eccentricity effect in contrast sensitivity.

Human visual performance for basic visual dimensions (e.g., contrast sensitivity and acuity) peaks at the fovea and decreases with eccentricity. The eccentricity effect is related to the larger surface area of the visual cortex corresponding to the fovea, but it is unknown if differential feature tuning contributes to this eccentricity effect. Here, we investigated two system-level computations underlying the eccentricity effect: featural representation (tuning) and internal noise. Observers (both sexes) detected a Gabor embedded in filtered white noise which appeared at the fovea or one of four perifoveal locations. We used psychophysical reverse correlation to estimate the weights assigned by the visual system to a range of orientations and spatial frequencies (SFs) in noisy stimuli, which are conventionally interpreted as perceptual sensitivity to the corresponding features. We found higher sensitivity to task-relevant orientations and SFs at the fovea than the perifovea, and no difference in selectivity for either orientation or SF. Concurrently, we measured response consistency using a double-pass method, which allowed us to infer the level of internal noise by implementing a noisy observer model. We found lower internal noise at the fovea than perifovea. Finally, individual variability in contrast sensitivity correlated with sensitivity to and selectivity for task-relevant features as well as with internal noise. Moreover, the behavioral eccentricity effect mainly reflects the foveal advantage in orientation sensitivity compared to other computations. These findings suggest that the eccentricity effect stems from a better representation of task-relevant features and lower internal noise at the fovea than at the perifovea.

Microsaccades and temporal attention at different locations of the visual field.

Temporal attention, the prioritization of information at specific points in time, improves performance in behavioral tasks but cannot ameliorate the perceptual asymmetries that exist across the visual field. That is, even after attentional deployment, performance is better along the horizontal than vertical meridian and worse at the upper than lower vertical meridian. Here we asked whether and how microsaccades-tiny fixational eye-movements-could mirror or alternatively attempt to compensate for these performance asymmetries by assessing temporal profiles and direction of microsaccades as a function of visual field location. Observers were asked to report the orientation of one of two targets presented at different time points, in one of three blocked locations (fovea, right horizontal meridian, upper vertical meridian). We found the following: (1) Microsaccade occurrence did not affect either task performance or the magnitude of the temporal attention effect. (2) Temporal attention modulated the microsaccade temporal profiles, and this modulation varied with polar angle location. At all locations, microsaccade rates were significantly more suppressed in anticipation of the target when temporally cued than in the neutral condition. Moreover, microsaccade rates were more suppressed during target presentation in the fovea than in the right horizontal meridian. (3) Across locations and attention conditions, there was a pronounced bias toward the upper hemifield. Overall, these results reveal that temporal attention benefits performance similarly around the visual field, microsaccade suppression is more pronounced for attention than expectation (neutral trials) across locations, and the directional bias toward the upper hemifield could reflect an attempt to compensate for typical poor performance at the upper vertical meridian.

Polar angle asymmetries in visual perception and neural architecture.

Human visual performance changes with visual field location. It is best at the center of gaze and declines with eccentricity, and also varies markedly with polar angle. These perceptual polar angle asymmetries are linked to asymmetries in the organization of the visual system. We review and integrate research quantifying how performance changes with visual field location and how this relates to neural organization at multiple stages of the visual system. We first briefly review how performance varies with eccentricity and the neural foundations of this effect. We then focus on perceptual polar angle asymmetries and their neural foundations. Characterizing perceptual and neural variations across and around the visual field contributes to our understanding of how the brain translates visual signals into neural representations which form the basis of visual perception.

Dissociable roles of human frontal eye fields and early visual cortex in presaccadic attention - evidence from TMS.

Shortly before each saccadic eye movement, presaccadic attention improves visual sensitivity at the saccade target 1-5 at the expense of lowered sensitivity at non-target locations 6-11 . Some behavioral and neural correlates of presaccadic attention and covert attention -which likewise enhances sensitivity, but during fixation 12 -are similar 13 . This resemblance has led to the debatable 13-18 notion that presaccadic and covert attention are functionally equivalent and rely on the same neural circuitry 19-21 . At a broad scale, oculomotor brain structures (e.g., FEF) are also modulated during covert attention 22-24 - yet by distinct neuronal subpopulations 25-28 . Perceptual benefits of presaccadic attention rely on feedback from oculomotor structures to visual cortices 29,30 ( Fig. 1a ); micro-stimulation of FEF in non-human primates affects activity in visual cortex 31-34 and enhances visual sensitivity at the movement field of the stimulated neurons 35-37 . Similar feedback projections seem to exist in humans: FEF+ activation precedes occipital activation during saccade preparation 38,39 and FEF TMS modulates activity in visual cortex 40-42 and enhances perceived contrast in the contralateral hemifield 40 . We investigated presaccadic feedback in humans by applying TMS to frontal or visual areas during saccade preparation. By simultaneously measuring perceptual performance, we show the causal and differential roles of these brain regions in contralateral presaccadic benefits at the saccade target and costs at non-targets: Whereas rFEF+ stimulation reduced presaccadic costs throughout saccade preparation, V1/V2 stimulation reduced benefits only shortly before saccade onset. These effects provide causal evidence that presaccadic attention modulates perception through cortico-cortical feedback and further dissociate presaccadic and covert attention.

Asymmetries in the discrimination of motion direction around the visual field.

The discriminability of motion direction is asymmetric, with some motion directions that are better discriminated than others. For example, discrimination of directions near the cardinal axes (upward/downward/leftward/rightward) tends to be better than oblique directions. Here, we tested discriminability for multiple motion directions at multiple polar angle locations. We found three systematic asymmetries. First, we found a large cardinal advantage in a cartesian reference frame - better discriminability for motion near cardinal reference directions than oblique directions. Second, we found a moderate cardinal advantage in a polar reference frame - better discriminability for motion near radial (inward/outward) and tangential (clockwise/counterclockwise) reference directions than other directions. Third, we found a small advantage for discriminating motion near radial compared to tangential reference directions. The three advantages combine in an approximately linear manner, and together predict variation in motion discrimination as a function of both motion direction and location around the visual field. For example, best performance is found for radial motion on the horizontal and vertical meridians, as these directions encompass all three advantages, whereas poorest performance is found for oblique motion stimuli located on the horizontal and vertical meridians, as these directions encompass all three disadvantages. Our results constrain models of motion perception and suggest that reference frames at multiple stages of the visual processing hierarchy limit performance.